24 research outputs found
LLMSTEP: LLM proofstep suggestions in Lean
We present LLMSTEP, a tool for integrating a language model into the Lean
proof assistant. LLMSTEP is a Lean 4 tactic that sends a user's proof state to
a server hosting a language model. The language model generates suggestions,
which are checked in Lean and displayed to a user in their development
environment. We provide a baseline language model, along with code for
fine-tuning and evaluation to support further development. We provide server
implementations that run on CPU, a CUDA GPU, or a Google Colab notebook, as a
step towards fast, effective language model suggestions for any user
MLE-guided parameter search for task loss minimization in neural sequence modeling
Neural autoregressive sequence models are used to generate sequences in a
variety of natural language processing (NLP) tasks, where they are evaluated
according to sequence-level task losses. These models are typically trained
with maximum likelihood estimation, which ignores the task loss, yet
empirically performs well as a surrogate objective. Typical approaches to
directly optimizing the task loss such as policy gradient and minimum risk
training are based around sampling in the sequence space to obtain candidate
update directions that are scored based on the loss of a single sequence. In
this paper, we develop an alternative method based on random search in the
parameter space that leverages access to the maximum likelihood gradient. We
propose maximum likelihood guided parameter search (MGS), which samples from a
distribution over update directions that is a mixture of random search around
the current parameters and around the maximum likelihood gradient, with each
direction weighted by its improvement in the task loss. MGS shifts sampling to
the parameter space, and scores candidates using losses that are pooled from
multiple sequences. Our experiments show that MGS is capable of optimizing
sequence-level losses, with substantial reductions in repetition and
non-termination in sequence completion, and similar improvements to those of
minimum risk training in machine translation
A Survey of Deep Learning for Mathematical Reasoning
Mathematical reasoning is a fundamental aspect of human intelligence and is
applicable in various fields, including science, engineering, finance, and
everyday life. The development of artificial intelligence (AI) systems capable
of solving math problems and proving theorems has garnered significant interest
in the fields of machine learning and natural language processing. For example,
mathematics serves as a testbed for aspects of reasoning that are challenging
for powerful deep learning models, driving new algorithmic and modeling
advances. On the other hand, recent advances in large-scale neural language
models have opened up new benchmarks and opportunities to use deep learning for
mathematical reasoning. In this survey paper, we review the key tasks,
datasets, and methods at the intersection of mathematical reasoning and deep
learning over the past decade. We also evaluate existing benchmarks and
methods, and discuss future research directions in this domain.Comment: Accepted to ACL 2023. The repository is available at
https://github.com/lupantech/dl4mat